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The 2009 Counting On program has evolved from a series that began in 1999 and which 
continued to expand and change until the current manifestation. The program has always 
had a twin learning focus upon both students and teachers. Thus it seeks to improve student 
mathematical outcomes while building capacity within the teachers by improving their 
professional situated mathematical knowledge which is the knowledge teachers need to 
effectively teach the early mathematical concepts to their students in a classroom context. 
Counting On 2009 was evaluated and this paper will use the findings of the evaluation 
report (White, 2010 in press) to examine whether the program was successful in changing 
student learning outcomes. 



The 2009 Counting On program has evolved from a series of Counting On programs 
that continued to expand and change into the current manifestation. The program has 
always had a twin learning focus upon both students and teachers. Thus it seeks to improve 
student mathematical outcomes while building capacity within the teachers by improving 
their professional situated mathematical knowledge which is the knowledge teachers need 
to effectively teach the early mathematical concepts to their students in a classroom 
context. 

This evolutionary process that started in 1999 saw the inclusion of a greater range of 
students (from Year 7 to Years 4 - 9), the inclusion of the feeder primary schools, and also 
a change in content and process. Until 2007 many of the basics of the diagnostic 
assessment had tended to remain essentially the same. Thus students were individually 
interviewed and videotaped for further analysis of their responses and this was a very time 
intensive process. In 2007 the program underwent significant modification that included a 
simplified assessment instrument and sorting process, where the interview was reserved for 
only the targeted students. There was also the introduction of Newman's Error Analysis; a 
revised Counting On CD to disseminate information and resources; the formation of 
School clusters; the use of a facilitator's conference; and a facilitated professional 
development model. The program began with and has continued to operate using a team 
approach and more recently school clusters have become Learning Communities and the 
globally successful Lesson Study model has been promoted as a structured way for 
Learning Communities and for members of a school team to work together. There was also 
a Counting On website. 

The Counting On program sought to address the concerns of the numeracy and literacy 
outcomes of school students detailed in the State Numeracy Plan 2006 - 2008 (NSWDET, 
2005a) and the State Literacy Plan 2006-2008 (NSWDET, 2005b) as well as the concerns 
listed under the six priority areas of the Office of Schools Plan 2009-11 (NSWDET, 2008), 
by building capacity among teachers while improving the student understanding of early 
mathematical concepts and procedures. 



Theoretical Basis. 



There has been considerable research completed since 1990 in children's early 
mathematical understanding. For instance, research into mental computation has revealed a 
rich and complex range of mental strategies that children develop for multi-digit addition 
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and subtraction tasks. Thus a sample of the available strategies for answering 47+18 could 
include: jump (47+18: 47+10 -* 57+3 60+5 -> 65), split (47+18: 40+10 = 50, 7+8 = 15, 

50+15 = 65), and compensation (47+18: 47+20 — > 67-2 65), where the use of these or 

other mental strategies involves a broad knowledge of number relationships. This complex 
maze of relationships within early mathematical learning has resulted in a number of early 
numeracy programs all containing frameworks such as the Victorian Early Numeracy 
Research Project (ENRP), the New Zealand Numeracy Development Project (NDP) and 
the New South Wales Count Me In Too (CMIT) program. What these numeracy learning 
frameworks have in common is a strong link between research, pedagogy, teacher 
professional learning and a strong focus upon the learning of the student. 

The research base for the program is closely related to the Counting On Numeracy 
Framework (Thomas, 1999) which was an extension of work by Cobb and Wheatley 
(1988), Beishuizen (1993), Jones, Thornton, Putt, Hill, Mogill, Rich and van Zoest (1996) 
and relates to the Count Me In Too Learning Framework in Number (LFIN) (Wright, 
1998; Wright, Martland, & Stafford, 2000). 

This research base was further supported by an increasing number of Counting On 
evaluation studies. Mulligan (1999) evaluated a pilot study involving 9 schools, after 
which the Counting On program began in 2000 with 40 schools, more than 600 students, 
120 school teachers and 40 district mathematics consultants. Further evaluation reports on 
the Counting On program were conducted in 2000, 2002, 2003 and 2007 (Perry & Howard, 
2000, 2002a, 2003; White 2008, 2009). 

The inclusion of Newman’s Error Analysis (NEA, Newman, 1977; 1983) in the 2007 
program aimed to assist teachers when confronted with students who experienced 
difficulties with mathematical word problems. Rather than give students more drill and 
practice, NEA provided a framework for considering the reasons for the difficulties and a 
process that assisted teachers to determine where misunderstandings occurred and where to 
target effective teaching strategies to overcome them. Moreover, it provided excellent 
professional learning for teachers and made a nice link between literacy and numeracy. 

Newman (1977, 1983) maintained that when a person attempted to answer a standard, 
written, mathematics word problem then that person had to be able to pass over a number 
of successive hurdles: Reading (or Decoding), Comprehension, Transformation, Process 
Skills, and Encoding. Along the way, it was always possible to make a careless error and 
there were some who gave incorrect answers because they were not motivated to answer to 
their level of ability. While there are many other theoretical approaches available to 
teachers, NEA offers one of the easiest to use and adapt and has proven popular among 
teachers for both the ease of the diagnostic features and also because it is easily used as 
classroom pedagogical and problem solving strategies. 

This brief and far from comprehensive overview has sought to portray the 2009 
Counting On program as an initiative that arose from an initial successful trial program that 
has continued to adapt and evolve each year to meet the changing challenges, concerns and 
demands of the students, teachers and system. There were changes incorporated into the 
2009 program but it is beyond the scope of this paper to present the evaluation of the 
whole program and this paper will report only on the success of the 2009 program in 
overall terms of student mathematical learning outcomes. 

Methodology 

The 2009 program was implemented in 88 schools across the state. The schools were 
divided into 21 Learning Communities and each community was assisted by a Regional 
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Mathematics Consultant. In each school there was a teacher with the title of School 
Program Facilitator who formed a team of teachers to implement the program. In each 
school, the facilitator coordinated the process whereby each teacher administered a whole 
class assessment test covering place value, addition, subtraction, multiplication, division 
tasks and word problems. The assessment test was closely li nk ed to the learning 
framework. These data were used by the teacher to identify the student target group. Target 
students scored few or no correct answers. The target group was then interviewed. Using a 
pre-test post-test procedure, on two occasions, teachers were asked to conduct a target 
group assessment process with a minimum of 5 students per class and facilitators were 
asked to record the student data on an excel spreadsheet supplied to them. The spreadsheet 
recorded the initial level on the LFIN and NEA items for the targeted students before the 
2009 Counting On program was implemented and again following 10 weeks of targeted 
2009 Counting On activities. These results were compiled and are reported in the next 
section. 



Results 

A total of 69 schools (78%) submitted data during September, consisting of 52 primary 
schools, 15 secondary schools and 2 special schools. There were 945 students included on 
the spreadsheet with 618 primary students (65.4%) and 327 secondary students (34.6%). 

Table 1 

Target Student Numbers in Each School Year 



School Year 


Frequency 


Percentage Frequency 


4 


19 


2.0% 


5 


330 


34.9% 


6 


269 


28.5% 


7 


207 


21.9% 


8 


110 


11.6% 


9 


10 


1.1% 


Total 


945 


100.0% 



Place Value 

In Table 2 below the initial and final LFIN levels of the 945 students are displayed for 
place value and a comparison of levels indicates an increase in the overall results from 
initial to final. There were 17% of students initially identified at the lowest level and that 
was reduced to 3% by the end of the program. 



Table 2 

The Initial and Final Place Value Levels 



PV Levels 


Initial 

Frequency 


Percentage 

Frequency 


Final 

Frequency 


Percentage 

Frequency 


0 


162 


17.14% 


30 


3.17% 


1 


352 


37.25% 


189 


20.00% 


2 


335 


35.45% 


457 


48.36% 
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3 


68 


7.20% 


173 


18.31% 


4 


21 


2.22% 


66 


6.98% 


5 


7 


0.74% 


30 


3.17% 


Total 


945 


100.00% 


945 


100.00% 


Table 3 shows the degree of student differences in levels between the initial and final 
levels for place value. It shows that the majority of students have improved by 1 or more 
levels (58.1%), with a sizeable group improving two levels (13.0%). A small number of 
students improved by 3 and 4 levels, and a small number declined by 1 or 2 levels. 


Table 3 

The Difference in Place Value Levels 






Difference 




Frequency 




Percentage Frequency 


-2 




2 




0.2% 


- 1 




12 




1.3% 


0 




382 




40.4% 


1 




409 




43.3% 


2 




123 




13.0% 


3 




16 




1.7% 


4 




1 




0.1% 


Total 




945 




100% 



The descriptive statistics record an increase in the mean from 1 .42 for the initial level 
(SD = 0.977) to 2.15 for the final level (SD = 1.026). Using a paired sample T-Test, the 
results indicate that the improvement in the student place value learning outcome levels at 
the start and finish of the 10 week 2009 Counting On program was statistically significant. 

Multiplication and Division 

Table 4 displays the initial and final LFIN levels for multiplication / division for the 
945 students and also indicates an increase in the overall levels. It shows an overall 
improvement in the levels and where initially there were nearly 15% of students identified 
at the highest level, this increased to 35% by the completion of the program. When the 
differences in levels are further examined in Table 5 they show that the majority of 
students have improved by 1 or more levels (57.6%), with a sizeable group improving two 
levels (13.8%). A small number of students improved by 3 and 4 levels, and a small 
number declined by 1, 2 or more levels. 

The descriptive statistics record an increase in the mean from 2.93 for the initial level 
(SD = 1.323) to 3.77 for the final level (SD = 1.186). Using a paired sample T-Test, the 
results indicate that the improvement in the student multiplication / division learning 
outcome levels at the start and finish of the 10 week 2009 Counting On program was 
statistically significant. 

Table 4 

The Initial and Final Multiplication/Division Levels 

PV Levels Initial Level Percentage Final Level Percentage 

Frequency Frequency Frequency Frequency 
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1 175 

2 198 

3 228 

4 206 

5 138 


18.52% 

20.95% 

24.13% 

21.80% 

14.60% 


42 

124 

175 

274 

330 


4.44% 

13.12% 

18.52% 

28.99% 

34.92% 


Total 945 


100.00% 


945 


100.00% 


Table 5 


The Difference in Multiplication/Division Levels 




Difference 


Frequency 




Percentage Frequency 


-3 


2 




0.2% 


-2 


3 




0.3% 


- 1 


26 




2.8% 


0 


370 




39.2% 


1 


341 




36.1% 


2 


130 




13.8% 


3 


63 




6.7% 


4 


10 




1.1% 


Total 


945 




100.00% 



Mathematical Word Problems - Newman 's Error Analysis 

While there were two questions used involving Newman’s Error Analysis in the 
assessment instrument' only the NEA result for the 'Natalie paddling the Murray River' 
item in both the initial and final assessments were recorded for each student. The NEA 
scale from 1 to 5 was used with a category 6 added to represent those who could complete 
the word problem successfully. 



Table 6 

The Initial and Final Newman's Error Analysis Levels 



NEA 

Levels 


Initial Level 
Frequency 


Percentage 

Frequency 


Final Level 
Frequency 


Percentage 

Frequency 


1 


142 


15.03% 


63 


6.67% 


2 


352 


37.25% 


202 


21.38% 


3 


279 


29.52% 


291 


30.79% 


4 


115 


12.17% 


228 


24.13% 


5 


29 


3.07% 


78 


8.25% 


6 


28 


2.96% 


83 


8.78% 


Total 


945 


15.03% 


945 


100.00% 



Table 6 displays the initial and final NEA levels and indicates an improvement in the 
overall levels from the initial to the final student assessments. When explored further, 
Table 7 shows that the majority of students have improved by 1 or more levels (53.8%), 
with a sizeable group improving two levels (15.6%). Initially there were 67% of students 
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identified as experiencing difficulties with the two NEA levels of Comprehension and 
Transformation and this was reduced to 52% by the end of the program. A small number of 
students improved by 3 and 4 levels, and a small number declined by 1, 2 or more levels. 



Table 7 

The Difference in Newman's Error Analysis Levels 



Difference 


Frequency 


Percentage Frequency 


-4 


2 


0.2% 


-3 


2 


0.2% 


-2 


6 


0.6% 


- 1 


45 


4.8% 


0 


382 


40.4% 


1 


317 


33.5% 


2 


147 


15.6% 


3 


32 


3.4% 


4 


12 


1.3% 


Total 


945 


100.00% 



The descriptive statistics record an increase in the mean from 2.60 for the initial level 
(SD = 1.151) to 3.32 for the final level (SD = 1.319). Using a paired sample T-Test, the 
results indicate that the improvement in the student outcomes for mathematical word 
problem levels at the start and finish of the 10 week 2009 Counting On program was 
statistically significant. There is a difficulty here in that these statistics rely on the 
assumption of the NEA levels being either a ratio or interval scale which is questionable 
regarding the equality of the distances between any two of the levels. 

Discussion 

The 2009 Counting On program had a positive impact upon students’ early 
mathematics learning through its twin learning focus upon both students and teachers 
There is a growing body of evidence claiming that teacher quality is one of the most 
important school factors influencing student achievement, ahead of class size and school 
size (Darling-Hammond, 2003; Cuttance, 2001). It is proposed but not proven that the 
student learning outcomes improved partly because teachers, through the support and 
resources of the program, had the opportunity to think, plan and reflect on their teaching, 
which produced a wider range of classroom strategies and a greater use of concrete 
materials. 

The full effects of improved teacher professional learning are often delayed and reveal 
themselves later as the teacher completes the process of integrating the new learning into 
current practice. So while it may be impossible to measure the full effects at this time, it is 
possible to gather some indicators. The data collected indicated that a statistically 
significant improvement existed in student learning outcomes in all three specific areas 
measured. It is argued that repetition of the test would not influence the results as the 
students received no feedback on the initial test and there was at least a ten week gap 
between assessments. The use of a testing procedure raises the issue of whether a correct 
answer equates to understanding and that tests do not necessarily reflect their level of 
understanding of mathematical concepts and relationships (Ellerton & Olson, 2005). 
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Research has indicated a 35% mismatch with students who gave correct answers with little 
or no understanding and others who gave incorrect answers but possessed some 
understanding. While these findings cast doubt on the use of large scale testing programs 
as a means of making comparisons or being used as basis for the allocation of resources, it 
is less of an issue for this program as the groups of targeted students are small for each 
school and teachers make use of instruments LFIN and NEA which are designed to assist 
teachers in diagnosing the level of student understanding. 

In a short program such as this, the results are quite remarkable. For some students, it is 
unrealistic to expect they will register an immediate improvement as they have been 
struggling for some time with their mathematical and literacy levels and have developed 
judgements of their own ability. To improve one level on either the LFIN or the NEA scale 
in such a small time frame is quite remarkable and points to educational significance. 
There is an expectation that these gains will continue as the students build upon their 
success and a longitudinal study of these students would be of interest but is beyond the 
scope of this paper. 

There are alternative reasons for a lack of student progress or in some cases a 
regression in the levels. Some students have become fixated on inefficient correct 
procedures while others have 'fossilised' misconceptions (Vaiyatvutjamai & Clements, 
2004). Also, the 2007 evaluation report explored reasons for the negative regression and 
listed factors such as the use of different assessors, poor initial teacher understanding of the 
LFIN and NEA, misdiagnosis, student resistance to assessment, and teacher confusion with 
the different levels for LFIN and NEA. It appeared that the errors originated from the same 
small number of facilitators and suggested inexperience and lack of understanding with the 
instalments. 

This paper concludes that the 2009 Counting On program was successful in assisting 
the learning outcomes of middle years students who struggled with their early mathematics 
knowledge. The author wishes to acknowledge the support of the New South Wales 
Department of Education and Training, particularly Peter Gould, Chris Francis and Ray 
MacArthur of the Curriculum Support Directorate. The opinions expressed in this paper 
are those of the author and do not necessarily reflect those of the Department. 
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